ADR-0001: Scope compliance claims to implemented controls - #1
Merged
Conversation
Removes the SOC 2 Type II, HIPAA, and GDPR assertions from README.md and replaces them with design-stage language, since no audit artifact exists and the compliance service has never run against a real schema. Deletes "BAA management" outright: a Business Associate Agreement is a signed legal contract, not a software feature. Retains the encryption claim, which is genuinely implemented, with a pointer to the Terraform resources. Marks soc2-evidence-collection.yaml as a design specification and flags its 10 unresolvable evidence paths with TODO. Adds two CI checks so this cannot regress: check_compliance_claims.py fails on a compliance claim not backed by a resolvable link or a design-stage qualifier, and check_evidence_paths.py fails on an unmarked evidence path that does not exist. Implements copilot-agent/docs/adr/ADR-0001-compliance-claims-remediation.md
Adds the compliance-claims job the ADR's Verification section calls for. It was written and validated with the original change but could not be pushed then: the token lacked workflow scope. Uses needs['compliance-claims'].result rather than dot access in the ci-success result check -- a hyphenated job id is parsed as subtraction in a GitHub expression, so the dot form would not have evaluated correctly. Implements copilot-agent/docs/adr/ADR-0001-compliance-claims-remediation.md
Removes compliance-claims from the ci-success needs list. That umbrella job is currently red on main for unrelated reasons -- format, lint, security, test and coverage all fail there -- so nesting a passing documentation check inside it masked a meaningful signal behind debt the check cannot influence. The job still runs on every push and pull request and reports its own status. If it should block merges, make it a required check in branch protection, which gates on it directly rather than through an aggregate. This also makes the ci.yml change purely additive: the ci-success block is now byte-identical to main. Implements copilot-agent/docs/adr/ADR-0001-compliance-claims-remediation.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
docs/adr/ADR-0001-compliance-claims-remediation.md(Phase 1 — documentation-only). The ADR itself is included in this PR since it was not previously committed.Why
README.mdpresented SOC 2 Type II, HIPAA, and data residency as delivered capabilities. They are not. There is no audit artifact, and the compliance service backing those claims has never successfully served a request — all 15 tables it queries are undefined in the only migration that exists. Representing unaudited software as SOC 2 Type II certified is a misrepresentation independent of code quality, and HIPAA language specifically invites healthcare customers to route PHI into a system whose PHI logging currently throws on every call.The engineering work behind these claims is real and competent; the README overstates its status. This PR corrects the status only. No
services/compliance/code is touched — the technical fix is ADR-0002.What changed
README.md:33-36— section retitled "Compliance & Governance (Design Stage)". SOC 2 reworded to "SOC 2 control mapping and evidence-collection design (no audit performed)"; HIPAA to "HIPAA-oriented control catalog covering §164.308/310/312 (design stage, not operational)"; Data Residency to "Data residency policy model (design stage)".:208— compliance service description reworded to design-stage, marked not deployed, linked to the ADR.:210-216— added aStatuscolumn, every rowDesign.:35and:214, not reworded. A Business Associate Agreement is a signed contract under §164.502(e), not a software feature; to a healthcare buyer "BAA management" reads as "Agentics will sign a BAA.":218— note above the endpoint list stating the service is deployed in no environment.:470— Phase 5 row split into "AI/ML Platform — Complete" and "Compliance — Design Complete, Implementation In Progress".:473-480— section retitled "Security"; SOC 2 Type II, HIPAA, and GDPR bullets deleted. Encryption retained unchanged — it is genuinely implemented — now citingdeployment/terraform/main.tf:243,298-299,340-344. I verified all five of those line citations resolve to realstorage_encrypted,at_rest_encryption_enabled,transit_encryption_enabled, and SSE-KMS settings before asserting them.deployment/compliance/soc2-evidence-collection.yaml— header markedStatus: Design Specification -- no audit performed; the 10 unresolvablepath:entries marked# TODO: artifact does not yet exist. I resolved all 14 paths mechanically; exactly the 10 named in the ADR are missing and the other 4 resolve, confirming Finding 7.New:
scripts/check_compliance_claims.py— ADR Verification 2. GrepsREADME.mdanddocs/forSOC 2|SOC2|HIPAA compliant|BAA|GDPR compliant|certifiedand fails unless each hit is within 3 lines of a resolvable repo path or a design-stage qualifier.New:
scripts/check_evidence_paths.py— ADR Verification 3. Resolves everypath:in the evidence manifest, fails on any that neither exists nor is marked# TODO.Test output — real, not fabricated
Both checks pass on this branch, and both were verified to actually fail when a regression is introduced (not just to pass trivially):
soc2-evidence-collection.yamland.github/workflows/ci.ymlwere both re-parsed with a YAML parser after editing to confirm they remain valid.Deviations from the ADR — please review these three
1. The claims check ships with a 14-entry baseline, and I did not edit the docs it flags.
ADR Verification 2 specifies the check covers
README.mdanddocs/. Running it repo-wide surfaced 14 pre-existing unbacked claims outside the ADR's enumerated edit list, six of which are substantively worse than anything in the README:docs/specification.md:776soc2: Type II certifieddocs/specification-summary.md:119SOC 2 Type II: Certifieddocs/architecture/DELIVERABLES.md:168SOC 2 Type II: Certifieddocs/architecture/DELIVERABLES.md:668SOC 2 Type II certificationdocs/specification.md:1070SOC 2 Type II compliancedocs/architecture/DATA_STORAGE_DELIVERABLES.md:495Compliance: GDPR, SOC 2, HIPAA compatibleI had three options and took the third: rewrite six documents the ADR did not scope; weaken the check until it passes; or enable the check as specified with an explicit in-tree baseline (
docs/compliance-claims-baseline.txt). I did not rewrite them because these read as requirement statements in specification documents — the surrounding lines say "Full compliance required", "Mitigation required" — and ADR-0001 Finding 9 shows the ADR author reviewed doc status deliberately. Reclassifying six design docs is a judgment call that belongs to whoever owns them, not to this PR.The baseline is matched on file + exact line text, not line number, so editing any baselined line re-surfaces it as new and it cannot silently absorb a fresh claim. It is a visible backlog, not a hidden exemption; the header says so. If you would rather I fix those six lines now, say so and I will. I flag
specification.md:776in particular:soc2: Type II certifiedsitting next tohipaa: Not supported (medical data prohibited)is a directly contradictory pair worth a human decision.2. The endpoint line for BAA was reworded, not deleted. The ADR says delete "BAA management" from
:35and:214(done, both deleted).:223is an endpoint listing for a route that genuinely exists, so deleting it would make the API docs wrong. I reworded it to "BAA registry records, design stage (tracking only; does not execute agreements)" — the registry/execution distinction the ADR's Decision item 3 draws. Say the word if you want it gone entirely.3. CI wiring is NOT in this PR and must be applied by hand. I wrote and validated the workflow job, but the push was rejected:
refusing to allow a Personal Access Token to create or update workflow .github/workflows/ci.yml without workflow scope. The two checks therefore do not run in CI until someone withworkflowscope adds this. Add to.github/workflows/ci.yml:and add
compliance-claimsto theci-successjob'sneeds:list and its result check. (Verified locally: the editedci.ymlparses and yields 12 jobs.)Migration execution status
Not applicable — this PR contains no migration and no code change. Nothing was executed against any database. The schema work is ADR-0002.
Left undone (by design — ADR Phases 2-4)
Everything in ADR-0001 Phases 2, 3, and 4 is out of scope here; Phase 1 was specified to ship independently. Phases 2-3 (the 15-table migration, auth on the compliance routers, deterministic
pseudonymize(), fail-closed key loading, enforced PHI logging) are ADR-0002, in a separate PR. Phase 4 — policy artifacts, a third-party §164.308(a)(1)(ii)(A) risk analysis, counsel-produced BAA template, and an independent CPA firm's Type II engagement — is not engineering work and cannot be done in a PR.Per ADR Verification 6, SOC 2 Type II language may return to
README.mdonly alongside an auditor's report reference, and HIPAA/BAA language only alongside the risk analysis and BAA template. This PR does not unblock either.